CMP-3883: Create an e2e suite for operator deployment tests - #1110
Conversation
|
@taimurhafeez: This pull request references CMP-3883 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
🤖 To deploy this PR, run the following command: |
|
/retest-required Looks like CI setup failed |
|
@taimurhafeez: This pull request references CMP-3883 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/retest-required |
|
This might be a silly question, but how do these tests do something different than the ones they were copied from? I'm struggling to figure out how it knows to only test deployment related things rather than what they were testing in their original files. Also, I noticed that the ticket itself mentions isolating tests into their own suite, does that mean that the tests these were copied from should be removed? |
rhmdnd
left a comment
rhmdnd
left a comment
There was a problem hiding this comment.
Only minor comments that could be addressed in a follow on. Once we add a new suite in openshift/release for these and start running them in CI, I think we can clean up the old ones. That way we ensure coverage through the migration.
Thanks!
| timeouterr := wait.Poll(framework.RetryInterval, framework.Timeout, func() (bool, error) { | ||
| pb := &compv1alpha1.ProfileBundle{} | ||
| if lastErr := f.Client.Get(bctx, pbkey, pb); lastErr != nil { | ||
| log.Printf("error getting ocp4 PB. Retrying: %s\n", err) |
| E2E_GO_TEST_FLAGS?=-v -test.timeout 120m | ||
|
|
||
| # By default we run all tests; available options: all, parallel, serial | ||
| # By default we run all tests; available options: all, parallel, config, deployment, serial |
There was a problem hiding this comment.
Is the config suite a separate PR?
There was a problem hiding this comment.
|
|
||
| .PHONY: e2e-deployment | ||
| e2e-deployment: e2e-set-image prep-e2e ## Run operator deployment end-to-end tests concurrently. | ||
| @CONTENT_IMAGE=$(E2E_CONTENT_IMAGE_PATH) BROKEN_CONTENT_IMAGE=$(E2E_BROKEN_CONTENT_IMAGE_PATH) $(GO) test ./tests/e2e/deployment $(E2E_GO_TEST_FLAGS) -args $(E2E_ARGS) | tee tests/e2e-test.log |
There was a problem hiding this comment.
This was my fault initially in that i reused the same log file for all logs, but it might make sense to break these into their own log files now that we're going to have more suites (e.g., tests/e2e-deployment-tests.log)
|
🤖 To deploy this PR, run the following command: |
My understanding is that there is no change regarding what will be tested with these tests. It is merely an reorganization. The reason for creating suites is to have ability to run these in parallel. Why now is the better time to do that is, cause we are moving downstream test cases upstream. If all suites are created before other downstream ported PR land, all those tests will have a suitable place to reside. Yes, as Lance has suggested, the plan is to remove these from original ones once CI with new suites is functioning as expected. Hope I comprehended and answered correctly. |
|
🤖 To deploy this PR, run the following command: |
| e2e-parallel: e2e-set-image prep-e2e ## Run non-destructive end-to-end tests concurrently. | ||
| @CONTENT_IMAGE=$(E2E_CONTENT_IMAGE_PATH) BROKEN_CONTENT_IMAGE=$(E2E_BROKEN_CONTENT_IMAGE_PATH) $(GO) test ./tests/e2e/parallel $(E2E_GO_TEST_FLAGS) -args $(E2E_ARGS) | tee tests/e2e-test.log | ||
|
|
||
| .PHONY: e2e-deployment |
There was a problem hiding this comment.
Good deal - this is being wired up in https://github.com/openshift/release/pull/76333/changes#diff-b82cf05db3d6825e537d43e7630499543f1897657729631dcc60a5765989f3aeR131
|
Looks great and the |
|
🤖 To deploy this PR, run the following command: |
|
/retest |
1 similar comment
|
/retest |
|
@taimurhafeez: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/test e2e-aws-parallel |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rhmdnd, taimurhafeez, yuumasato The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
5 participants
Created a dedicated test suite for operator deployment functionality tests.
New Suite:
tests/e2e/deployment/3 deployment tests copied from parallel and serial suites
Tests focus on: metrics endpoints, HTTP versions, operator upgrade behavior
Run with:
make e2e-deploymentTests included:
TestServiceMonitoringMetricsTarget(Prometheus metrics monitoring)TestResultServerHTTPVersion(HTTP version verification)TestProfileBundleDefaultIsKept(operator upgrade/restart behavior)Tested on OCP 4.22.
--- PASS: TestServiceMonitoringMetricsTarget (74.15s)
--- PASS: TestProfileBundleDefaultIsKept (58.37s)
--- PASS: TestResultServerHTTPVersion (14.76s)
Assisted by Claude.